Process Assessment and Process Improvement - the need to Standardise?

During the last few years there has been considerable interest in software process assessment and improvement methods based on the Software Engineering Institute's(SEIs) pioneering work in this area. The emergence of a number of variations of the SEI's method suggests the need to standardise on an internationally approved method of software process assessment and improvement based on good management practices in software development. The paper suggests however that a greater need is for research into the issues concerned, rather in producing what would inevitably be a rather arbitrary standard. NEED FOR PROCESS ASSESSMENT AND IMPROVEMENT "It is estimated that in Britain £500 million per annum is wasted because of poor software design and development". [DTI]. Will the widespread adoption of software engineering methods and tools by the software development industry be successful in solving a problem of this magnitude? Ever since computer-aided software engineering (CASE) tools were developed, they have been an important means of improving productivity and quality. But success is subjective, and thus far has largely been defined by the developers. Therefore, both advocates and sceptics can reasonably demand a concrete measure of related improvement. An important initial step in addressing software problems or issues is to treat the entire development task as a process which can be controlled, measured, and improved. It is a matter for debate whether software quality is best measured directly from the software product itself or indirectly from the software process. The ESPRITII project SCOPE in which one of the authors is involved (RBH), is concerned with the former view which emphasises product measurement, whereas the software process assessment methods mentioned in this paper are more concerned with ensuring the quality of the software product by control of the corresponding process. It can be argued that, since the quality of the software product cannot be determined until late in the development cycle, reliance has to be placed on process quality as a means of achieving product quality. The internal process can be seen as the key to quality management yet most software contractors do not use a defined, repeatable and measurable software process.[13] Major international purchasers of software-based systems in the telecommunications, nuclear, avionics, and defence sectors have expressed the view that there is a general need to supplement the reliance on ISO 9001 as an indicator of good practice in the procurement process [18]. Procurement policies within Government are concerned to obtain "best value for money" where value is regarded as quality (fitness for purpose) and delivery against price. This has led some organisations to put into place software assessment schemes which demand demonstrable evidence of continuous quality improvement programmes and/or require that the software engineering capability of suppliers is assessed during the tender (called "bid" in the United States) evaluation process. Bill Curtis of the Software Engineering Institute (SEI) spoke of the need for PROCESS, PEOPLE and TOOLS (TECHNOLOGY) as three prongs of a concerted software productivity and quality effort. Software engineering has four aspects [21] * Process: the sequence of phases, tasks and activities required to develop and maintain software. This includes the life-cycle paradigm, whether it is the traditional waterfall model or a prototyping-based model. * Method: the approach to solving a problem. A method is a technique, an organised way of doing something, a procedure for performing a task. A method implies the existence of rules to be followed in the carrying out of the constituent parts of the activity. A method can apply to one or many phases of the life cycle, eg. Jackson Structured Programming (JSP) and Jackson System Development (JSD). It is often stated in CASE implementation: "No method, no hope". * Tools: support for processes and methods. Tools provide this support by performing repetitive tasks like tracking changes, analysing consistency and maintaining relationships. * People: many studies have shown that the single greatest factor in increasing productivity and quality is the skill and knowledge of the personnel employed in the software process. The process and method(s) must be determined before considering the role of tools, otherwise the process and methods become secondary to the tool. An overall tool strategy for an organisation or a project should consider the processes and methods in place, determine the functions in the life cycle that need to be supported and account for the environment infrastructure that will eventually let the tools be integrated. This approach is consistent with the trends towards greater portability, but it differs from most organisations' tactics, where a tool is chosen and then additional tools and the software engineering process and methods are forced into the framework of that tool. PROCESS ASSESSMENT AND IMPROVEMENT METHODS Introduction to concepts Before any attempt can be made to improve the software process by an organisation, the baseline of how software is currently developed must be defined. An understanding of where effort is put as well as types of errors typically encountered is essential. Without a clear understanding of the software process and product that typify the environment, there is no hope of improving or assessing the quality of software being generated. Although a 'quality' process does not guarantee a 'quality' product, a 'quality' process may be necessary but not sufficient to produce a 'quality' product. A general process assessment method contains a number of elements: a) a diagnostic tool or instrument to enable the maturity of software processes to be meansured and a maturity level or profile to be drawn up b) an underlying generic process maturity model on which to base the assessment and to derive the priorities for improvement c) a set of best practices or norms for comparison, either explicitly, or implicitly in the assignment of levels or profiles d) supporting best practice guidelines to support the improvement planning process. e) a clear mechanism for presenting results. An assessment instrument (usually a questionnaire) is used as an aid to data gathering. A good questionnaire is structured for ease of completion and capable of providing quantified responses. An assessment method should allow for two approaches to data gathering: a) self-assessment by a knowledgeable team, backed up if necessary through interview/audit by a specialist assessment/evaluation team. b) completion by a specialist team on the basis of structured interviews of staff within the organisation. Assessment is normally carried out at project level by looking at the processes used on past or current projects in order to capture actual practice. The capability to perform thenext project is based on the process maturity distribution of current projects. Large projects will have a lifecycle and processes defined specifically to meet the needs of the project. Different types of product, or different development modes may result in significantly different process profiles. Hence an effective assessment scheme needs flexibility to handle the variations. In terms of expressing the results, process assessment may give ratings for individual processes, process areas, projects or an overall rating for an organisation. In the latter case, a method is required to combine the results from possibly disparate projects, each with differing levels of maturity. One of the primary uses of process assessment is for process improvement. Assessment leads to the identification of weaknesses and the key areas for improvement and provides the input into improvement action plans. Another use is for capability assessment, which is linked with evaluation of suppliers. Figure1 Process improvement gives a visual representation of the major inputs, processes and outputs of process assessment when used for process improvement. Method or standard? A question currently being asked is "Is a method a standard and is a standard a method?" A comprehensive coverage of a range of methods and standards suitable for process assessment is given in [18]. The coverage is from generic standards for use in software production such as the International ISO 9000 series of standards to more software specific methods of process assessment. Let us now consider process standards. Process standards Software process standards are standards which constrain the process which may be used to develop and maintain software by defining characteristics or properties which a conforming process must possess. These standards are usually used to try and ensure that the software product produced by executing the process is of adequate quality. Process standards used in this context include the military standards DEF STAN 00-55 and NATO AQAP-13 and the civil standards IEEE 730 and ISO 9001. Most of these standards are specific to software the exception being ISO 9001, a generic standard intended for manufacturing processes in general, which has only comparatively recently been considered for application to software production. The ISO 9000 series of standards was based on BS 5750; a British standard which first emerged in 1979 and was itself based on DEF STANs 0521 and 05-24. More recently in 1987, BS 5750 was revised to incorporate improvements made by ISO, rendering the two standards essentially identical. Thus the current BS 5750 corresponds very closely to the ISO standards 9000 to 9004. Parts 2 and 3 of BS 5750 and the corresponding ISO standards 9002 and 9003 however are not normally considered appropriate for software, and in this section when we refer to the ISO 9000 series we will normally mean ISO 9001 which is equivalent to BS 5750 part 1. To further complicate the issue there is also a European name for the same standard, viz. EN 29001. Figure 2 helps to clarify the position. ISO 9001 is to be used for external quality assurance where it is necessary to demonstrate the implementation of quality elements for contractual, mandatory and assessment purposes. Unlike most software specific standards ISO 9001 is applied to the organisation itself and not to a specific project.